The Internet World Wide Web (“WWW”) has become so popular, and the traffic so heavy, that frustration with response times has led to jokes about the WWW standing for the World Wide Wait. Proxy servers, which were originally developed for security as extensions of firewalls, have proved to have additional value in reducing Internet access delays by increasing the speed with which their cached uniform resource locators (“URLs”) are returned to users.
Proxy servers, or “proxy caches”, allow the most requested URLs to be stored in cache. As a result, the proxy servers provide the following advantages and efficiencies:                Less network traffic. Once an object has been downloaded from the Internet, subsequent users will retrieve that object from the cache instead of having to request the same object across a remote network link.        Better user performance. Typically proxy caches reside much closer to an end user than the remote web server. Additionally, while requests across the Internet are serviced at lowest common denominator WAN speeds (typically 64 kbps-1.5 Mbps), requests to the cache are serviced at LAN speeds (10 Mbps-100 Mbps).        Better perceived network reliability. Having cached copies of objects compensates for transient Internet “brownouts” and loss of service due to heavy congestion on remote segments.        Caching has proven to be so efficient that the need was soon seen for deploying multiple proxy servers that could communicate and work together to create a more robust system. In 1995 the Internet Cache Protocol (“ICP”) was developed to allow individual proxies to “query” manually configured neighboring proxies in order to find cached copies of requested objects. If all queries failed to find a cached object, the proxy would then use HyperText Transfer Protocol (“HTTP”) to request the object from the Internet.        
Although ICP allows proxy servers to be networked together, certain problems emerge when using the protocol. These include:                ICP arrays must conduct queries to determine the location of cached information, an inefficient process that generates extraneous network traffic.        ICP arrays have “negative scalability” in that the more proxy servers added to the array, the more querying required between servers to determine location.        ICP arrays over a period of time tend to become highly redundant, with each server containing largely the same information—the URLs of the most frequently used sites.        